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ABSTRACT 



In this paper, a model is presented that uses educational 
dialogues to support the design process for learning support systems. The 
model is argued to provide a structure that links pedagogically motivated 
empirical studies of student- tutor-system interactions to problem 
specification and system design, and to result in the creation of learning 
technology systems capable of supporting learning. This model assumes an 
iterative life cycle model of software design, in which dialogue can be both 
an input (in the form of data for analysis) and an output (the result of 
learners using a well -designed system) . The model has been derived from 
analysis of patterns of system development, and is related to approaches to 
cyclic models in educational theory and research methods. Two contrasting 
studies are used as illustrative examples of this iterative life cycle. The 
paper concludes by presenting a comparison between this approach and related 
developmental models: Kolb's experiential learning and Action Research. 
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In this paper, a model is presented that uses educational dialogues to support the 
design process for learning support systems. This model assumes an iterative life 
cycle model of software design, in which dialogue can be both an input (in the 
form of data for analysis) and an output (the result of learners using a well- 
designed system). The model has been derived from analysis of patterns of 
system development, and is related to approaches to cyclic models in educational 
theory and research methods. Two contrasting studies are used as illustrative 
examples of this iterative life cycle. The paper concludes by presenting a 
comparison between this approach and related developmental models: Kolb’s 
experiential learning and Action Research. 





Introduction 

In this paper, a model that uses dialogue to support the design of systems for learning is 
presented. This model is argued to provide a structure that links pedagogically motivated 
empirical studies of student-tutor-system interactions to problem specification and system 
design, and to result in the creation of learning technology systems capable of supporting 
learning. Contrasting case studies are presented to support this claim. 

The importance attached to dialogue in this model stems in part from the work of 
Lipman (1991), who has argued that dialogue with a ‘community’, including teachers and 
other learners, can play an important role as part of an interactive learning mechanism. As 
well as articulating and explaining a concept to peers and tutor, the learner will also 
internalise these dialogues. Self-reflection on these concepts can be fine-tuned and regulated 
by reference to a history of external dialogues that the learner has been exposed to. 
Furthermore, new conceptual understanding can be ‘tested-out’ by exposing them to further 
external dialogue within the ‘community of learning’. 

The model described in this paper has emerged from an analysis of ongoing 
development work. Its proposed strength is based on the realisation that common structures 
appear in a diverse range of development activities. In particular, this methodological 
framework has been synthesised from developmental work in subjects as diverse as Musical 
Composition (Cook, 2000) and curriculum design (Oliver and Conole, 1998; Kewell et. al., 
1999b), demonstrating its transferability and validity. Our life cycle involves five steps: 1) 
problem specification, 2) educational dialogues and interactions are analysed and modelled, 
3) results from dialogue analysis can be used as the basis for the design of technology-based 
educational systems and/or can be used to inform our educational practice, 4) a system is 
implemented that can be used to foster informed or principled educational dialogues, and 5) 
the system is evaluated with users. There are various issues involved with each step: 

• At step 1 we need to decide if the application area is domain specific or more generic. 

• Step 2 requires us to decide on what dialogue analysis and modelling techniques to use. 

There is a great number of techniques available, a discussion of which is beyond the 
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scope of this paper (see http://www2.unl.ac.uk/-exbzcookj/workshop/workshop.htm for 
the proceedings from a workshop that included talks on some of the approaches that are 
possible). 

• It is not a question of direct transfer of expertise from human to machine, but rather to 
use human expertise, when modulated (rather than transferred) to the computational 
medium, as an appropriate starting point for system design, which can then be refined on 
the basis of evaluation with users. 

• When moving from design to implementation two possible choices are that of going for 
an implementation of the whole design in one go (the big bang) or of adopting an 
incremental approach that involves the repeated use of prototypes to elicit user 
requirements. In our model we favour the latter approach. 

• Evaluation can also be problematic. As has been discussed, a common problem with 
evaluation is that designer will adopt a familiar strategy, as opposed to the most 
appropriate strategy, and hence will not be able to address key issues. In addition, it may 
prove appropriate to use different evaluation strategies at different iterations within the 
cycle (Oliver, Conole et al., 1998). 

The purpose of this paper is not to provide solutions to all of the above issues. Rather, we 
make use of two studies to explore aspects of these design issues and develop a deeper 
understanding of this model. To this end, the paper concludes by presenting a comparison 
between this approach and related developmental models: Action Research and Kolb’s 
experiential learning. 



The first study: a support tool for musical creativity and reflection 

The first study used steps ‘1 through 5 and back to step 2’ of our life cycle to develop a 
learning support tool for musical creative reflection. Cook (1998) has used a dialogue 
analysis and modelling methodology to develop a computer-based pedagogical agent for 
supporting musical composition learning. Cook has explored the life cycle as follows: 

1. Problem specification. Evidence supports the view that the use of sequencer software in 
the current training of composers in higher education appears to limit creative thinking in 
some learners. 

2. Dialogue analysis and modelling. Cook (1998) has analysed teacher-learner dialogues in 
a formal manner; one outcome of this analysis being prescriptive models of interactions 
described at the level of participants’ goals and communicative acts (the latter being seen 
as a way of achieving goals). 

3. Design of technology-based educational systems. These dialogue models (generated by 
step 2) were then used in the design of an computer-based pedagogical agent called 
MetaMuse (Cook, 2000) that was able to engage in interactions that aimed to promote 
creative reflection. 

4. Implementation. MetaMuse, a pedagogical agent for supporting undergraduate musical 
composition learning, has been developed as a result of employing the design approach 
described above. 

5. Evaluation of the system with users. MetaMuse has been evaluated with users (Cook, 
2000). The sample size of users used in the evaluation, at 17, was small and so some 
caution is required when attaching significance to these results. Given this reservation, 
however, the results seem favourable: the total average response score for user attitudes 
to MetaMuse — elicited by questionnaire following a session with the system — was 



3.63 (out of 5). However, a question that asked users to evaluate the help screens and 
messages provided by MetaMuse received the lowest average score (2.76 out of 5). The 
help screens and the language used by MetaMuse clearly need improving. (Provision of 
help was not a focus for the initial version of MetaMuse.) 

Current work involves two related projects. The first project aims to improve the guidance 
and explanations provided by MetaMuse by incorporating model-based training techniques 
(new problem, step 1 of the life cycle). The goal of such an undertaking is to provide 
enhanced access to learning over the Internet in a domain (musical composition learning) that 
has traditionally had little computer-assisted support (i.e. training in musical composition). 
The second project involves an analyses of transcriptions of pairs of collaborating learners 
using MetaMuse (generated by the study described by Cook, 2000). The initial evaluation of 
the MetaMuse pedagogical agent (i.e. step 5 above) did not provide adequate insight into the 
following question (new problem, step 1 of the life cycle): what are the interactive means by 
which learning agents engage in cooperative problem-seeking? This second project will, 
therefore, commence from step 2 initiating a second loop around the life cycle (i.e. more 
dialogue analysis and modelling). 



The second study: pedagogic re-engineering 

In contrast to the first case study, the second project started from a different point in the 
model. This work concerns Media Adviser (Kewell et. al., 1999a), a software tool that 
supports “pedagogic re-engineering” (Nikolova and Collis, 1998): changing the structure of a 
course and the nature of its activities in order to take advantage of Learning Technology. The 
software supports a five-stage process of course description, analysis to identify strengths and 
weaknesses, identification of alternative teaching techniques, the creation of revised 
curriculum plan(s), and a planning process that identifies the steps needed to implement the 
new curriculum. Media Adviser comes into its own when used as a discussion artefact 
(Kewell et. al., 1999b), and consequently dialogue has been used as an integral part of the 
tool’s development (Oliver and Conole, 1998). 

The project involved implementing a software tool based on an existing design, and 
then testing this with users. This then led to the refinement of the software tool, involving an 
analysis of users’ dialogue, redesign and re-implementation. 

4. Implementation. The specification for the functionality of Media Advisor and a high-level 
design outline was given to the project team, who implemented a prototype. 

5. Evaluation of the system with users. The software was used as part of a professional 
development programme, in which users worked together to address problems of 
curriculum design. This process was observed and video recorded, and the data was 
subsequently transcribed. 

2. Dialogue analysis and modelling. The dialogues that were observed in the evaluation 
phase were analysed in order to identify strategies for use, potential refinements and 
unanticipated needs or issues. 

3. Design of technology-based educational systems. The software tool was refined and 
extended to address the findings of the analysis phase. This included enhancing the visual 
feedback on course profiles provided by the tool, increasing the flexibility of its 
modelling component, and allowing users to customise content with a ‘save and load’ 
facility. In addition, the navigation through the software was re-thought. 
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4. Re-implementation. Following the design of additional functionality and revisions to the 
interface, the software was re-implemented. 

In summary, then, this second study started at step 4 of the model, progressed to step 5, 
looped back to step 2, and then moved on through steps 3 and 4. Current work on this project 
involves a second round of empirical evaluation of the software tool. 

It is worth noting, however, that this project work built on existing research that can 
be mapped onto steps 1, 2 and 3 of the first cycle of the model. What this illustrates is that 
any research or design process is likely to draw on the full range of steps in the process, but 
that these may be carried out by separate researchers or project teams. Consequently, any 
self-contained team may only be aware of their progress through sub-sections of the model. 



Discussion 

The cases described above illustrate the model elaborated in this paper. However, it is useful 
to consider whether this model is actually innovative, or if it is simply a re-naming of an 
existing approach. To demonstrate the distinctiveness of this model, this section includes a 
comparison between it and two other models, each of which relates to a different aspect of its 
process: Kolb’s experiential learning and Action Research. 

Kolb's experiential learning cycle 

Just as systems design has shifted from system- to user-centred models, current theories of 
learning have moved from teacher-oriented to learner-oriented. Particularly relevant to this 
discussion is the model provided by Kolb’s experiential learning cycle (Kolb, 1984). This 
model can be taken to explain the process whereby the designer learns, through experience, 
how to improve their system. Importantly, the “experience” phase of this model relies on 
first-hand immersion in (or observation of) real use. 

This cycle fails to capture key elements of the model described in this paper. At a 
simple level, the mapping between stages (e.g. abstract reconceptualisation and design, 
experimentation and implementation, concrete experience and evaluation) highlights that the 
process in this paper is geared towards a practical process, rather than reconceptualisation. It 
is a process that deals with creation and development, as opposed to being purely concerned 
with the development of knowledge. 

Additionally, the design process involves a ‘scoping’ element (problem specification) 
that is missing from Kolb’s cycle. The experiential learning cycle assumes that the remit for 
learning is given; the model in this paper places the responsibility for this scoping with the 
designer. 

Finally, experiential learning also leaves the nature of “experience” undefined; in this 
model, a tighter focus is presented. The purpose of this is to concentrate on dialogue as a key 
indication of learning and development (e.g. Lipman, 1991). This reflects the aim of this 
model, which is to design systems that successfully support and structure learning through 
dialogue. 

Action Research 

Action Research is closely related to experiential learning, and follows a similar 
cyclic model (see, e.g., Zuber-Skerrit, 1992). The key difference between the two approaches 
can be argued to be one of emphasis. Experiential learning is closely linked to personal 
development (sometimes in the guise of Action Learning; e.g. Peters, 1998), particularly 
through reflective practice. It also focuses on the acquisition and development of concepts. 
By contrast, Action Research normally includes an element of dissemination (making it 
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partly a public process, rather than an exclusively private one) and is concerned with the 
identification and development of theories. In particular, it is viewed as one method of 
building grounded theories, and has been linked to Habermas conception of the organisation 
of enlightenment in critical communities (Zuber-Skerrit, 1992). 

This model described in this paper does not place such predetermined restrictions on 
the designer, but instead responds to their personal interests. As a consequence, the process 
could lead to either type of output, depending on the focus of the work and the context in 
which it is carried out. For example, immediate, local solutions can be created through the 
design process. In addition, more general schemas for design can evolve through the repeated 
application of this process, and through reflections on problems that emerge. 

One way of determining which of these emphases will emerge lies in the choice of 
evaluation strategy employed during the cycle. Traditional assumptions about transferability 
of evaluation findings suggest that highly authentic approaches will be best suited to local 
solutions, whilst controlled evaluations will support generalisations (Oliver et. al., 1998). 
Either approach could be adopted within this model. 

One commonality between all of the models discussed in this paper is the use of 
iteration. One of the strengths proposed for Action Research, for example, is that the process 
is incremental and iterative, leading to a process of continual improvement. A similar 
argument is made for the model outlined here. This is particularly true given its emphasis on 
users and learning, rather than systems and teaching, which lies at the heart of improving the 
usability of learning technology. 

However, one key difference between these models concerns the scope of their 
iterations. Action Research, as with experiential learning, assumes that iterations will involve 
complete repetitions of the four-stage process. The model proposed here is more versatile; in 
it, iterations may occur at one of four levels: 

1 . Re-implementation of the tool, necessitated by the identification of bugs or errors 

2. Revisions at the level of design, aimed at improving usability (a link back from 
evaluation to design) 

3. Reformulations that respond to unexpected patterns of use or to the recognition that 
current analytical techniques have not adequately explained the structure of the dialogue 
(a link back from evaluation to analysis) 

4. The identification of a completely new (but probably related) problem, requiring a 
distinct project that may replace or run in parallel with the original one (a link back to the 
start of the process) 



Conclusions 

In this paper, a range of cyclic models for development or the identification of good practice 
have been identified. A further cyclic model has been elaborated which focuses on the role of 
dialogue in design. This has been shown to be distinct from existing models, in particular in 
terms of the flexibility of its iterations and its focus. Two key features of this model are 
worth emphasising. 

Firstly, it is iterative in nature, which necessarily engenders a continuous process of 
research and development. Above we have described how our model has already investigated 
a limited set of cognitive and social processes that underpin the process of creative system 
design and innovation. 

Secondly, it illustrates how existing models can be adapted and customised in order to 
provide a detailed and relevant step-through for design processes in specific areas. Indeed, 
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because of the diverse domains presented in our two studies, we claim that our model has a 
high degree of transferability and validity. 

The current version of this model represents at least one iteration through a reflective 
cycle, such as that of Action Research, for the authors. It draws on the identified models, and 
on experiences of applying these in a research context. Reflections on these experiences 
identified refinements and changes, and also commonalities between practice in different 
research projects. These reflections led, in turn, to the model described above. This can be 
viewed as an attempt to develop a theory that can be used to explain and structure the design 
of systems. The next step will be to undertake further iterations of self-reflection in order to 
refine the model further. One aspect of this will be to consider how this model relates to 
project management and to the structure of research projects. The work will involve applying 
the refined model, both to further iterations of the cases described in this paper and to new 
projects, and then further developing the model based on these experiences. 
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